Papers with automatic and
DAMAGeR: Deploying Automatic and Manual Approaches to GenAI Red-teaming (2025.naacl-tutorial)
Copied to clipboard
| Challenge: | In this tutorial, we will review and apply current automatic and manual red-teaming techniques for GenAI models. |
| Approach: | This tutorial will review automatic and manual red-teaming techniques for GenAI models . |
| Outcome: | This tutorial will review and apply current automatic and manual red-teaming techniques for GenAI models. |
CodeDistiller: Automatically Generating Code Libraries for Scientific Coding Agents (2026.acl-demo)
Copied to clipboard
| Challenge: | Automated Scientific Discovery (ASD) systems rely on parametric knowledge to generate and run code-based experiments. |
| Approach: | They propose a system that distills large collections of scientific Github repositories into a vetted library of working domain-specific code examples. |
| Outcome: | The proposed system produces more accurate, complete, and scientifically sound experiments than an agent with only general materials-science code examples. |
Language Model Sentence Completion with a Parser-Driven Rhetorical Control Method (2024.eacl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate nonfacts and deviate from desired criteria for text generation. |
| Approach: | They propose a controlled text generation algorithm that enforces adherence toward specific rhetorical relations in an LLM sentence-completion context by a parser-driven decoding scheme. |
| Outcome: | The proposed method generates sentences that satisfy desired rhetorical relations in an LLM. |
Idiap NMT System for WAT 2019 Multimodal Translation Task (D19-52)
Copied to clipboard
| Challenge: | In the past few decades, multi-modality has received critical attention in translation studies, although the benefit of visual modality in machine translation is still in debate. |
| Approach: | They propose to use the Transformer model and IITB English-Hindi parallel corpus as additional data sources for the evaluation and challenge test sets. |
| Outcome: | The proposed system outperforms systems that consider visual information in the English-Hindi Multi-Modal Translation task. |
Unsupervised Aspect-Level Sentiment Controllable Style Transfer (2020.aacl-main)
Copied to clipboard
| Challenge: | Unsupervised style transfer has been explored in text. |
| Approach: | They propose a system where aspect-level sentiments can be controlled at the output . they propose to use unsupervised techniques such as ABSA masked-language-modelling . |
| Outcome: | The proposed system is successful in controlling aspect-level sentiments. |
Distractor Generation for Fill-in-the-Blank Exercises by Question Type (2023.acl-srw)
Copied to clipboard
| Challenge: | Existing studies have generated words that are semantically similar to the correct words as distractors for fill-in-the-blank questions. |
| Approach: | They propose a method to automatically generate distractors for fill-in-the-blank questions in entrance examinations for Japanese universities. |
| Outcome: | The proposed method is effective on 500 actual questions on English fill-in-the-blank questions in Japanese universities. |
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation (2021.naacl-main)
Copied to clipboard
Alexander Fabbri, Simeng Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq Joty, Dragomir Radev, Yashar Mehdad
| Challenge: | Recent advances in summarization are driven by the availability of large datasets such as the CNN-DailyMail corpus and the New York Times corpus. |
| Approach: | They propose a method for fine-tuning pretrained models for summarization in unsupervised manner . they use Wikipedia data to produce pseudo-summaries which contain characteristics of target dataset . |
| Outcome: | The proposed method achieves state-of-the-art, zero-shot abstractive summarization performance on CNN-DailyMail dataset and compares with other methods on other datasets. |
Courteously Yours: Inducing courteous behavior in Customer Care responses using Reinforced Pointer Generator Network (N19-1)
Copied to clipboard
| Challenge: | In order to ensure customer satisfaction and retention, it is imperative for customer care agents and chatbots to be cordial and emphatic to the customer. |
| Approach: | They propose a deep learning framework that automatically transforms neutral customer care responses into courteous replies by stylistic transfer. |
| Outcome: | The proposed model can generate courteous expressions consistent with the emotional state of the customer while preserving the content. |
Constructing Open Cloze Tests Using Generation and Discrimination Capabilities of Transformers (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing open cloze tests are laborious to design because they require a large number of variables to predict the distribution of words in a text passage. |
| Approach: | They propose a transformer-based model that exploits generation and discrimination capabilities to improve performance. |
| Outcome: | The proposed model outperforms previous work and baselines in 82% accuracy and can be used as a future benchmark. |
A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators (C18-1)
Copied to clipboard
| Challenge: | Existing research on visual question generation is focused on training models to fit the annotated data set that makes them indifferent from other language generation tasks. |
| Approach: | They propose to use two discriminators to enhance the training of a visual question generator to ask natural questions about an image. |
| Outcome: | The proposed model outperforms state-of-the-art models in terms of automatic and human evaluation metrics. |
Automatic Generation of Socratic Subquestions for Teaching Math Word Problems (2022.emnlp-main)
Copied to clipboard
| Challenge: | We hypothesize that questioning can enhance human performance and assist solvers . |
| Approach: | They propose to use large language models to generate sequential questions for math word problem-solving . they propose to apply these models to a variety of math word problems . |
| Outcome: | The proposed model improves the performance of a math word problem solver by generating more questions than other models. |
Automatic Detection of Fake News (C18-1)
Copied to clipboard
| Challenge: | a growing number of fake news detection tools are needed to identify trustworthy news sources. |
| Approach: | They propose to use two novel datasets to automate the identification of fake news . they propose learning experiments to build accurate fake news detectors . |
| Outcome: | The proposed algorithms achieve accuracies of up to 76% and compare them with other tools . the proposed algorithms are based on satirical news sources and fact-checking websites . |
Chinese WPLC: A Chinese Dataset for Evaluating Pretrained Language Models on Word Prediction Given Long-Range Context (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets for word prediction with long-range context have not been tested. |
| Approach: | They propose automatic and manual selection strategies tailored to Chinese to ensure that target words can only be predicted with long-term context. |
| Outcome: | The proposed model is 45 points behind human in terms of top-1 word prediction accuracy. |
ELF22: A Context-based Counter Trolling Dataset to Combat Internet Trolls (2022.lrec-1)
Copied to clipboard
| Challenge: | a new dataset aims to automate the method to counter trolls . trolleds cause psychological damage to individuals and increase social costs . |
| Approach: | They propose to use a dataset to generate counter responses by varying counter responses according to a given strategy. |
| Outcome: | The proposed method improves strategy-controlled sentence generation. |
Speaker or Listener? The Role of a Dialog Agent (2020.findings-emnlp)
Copied to clipboard
| Challenge: | chitchat bots are designed as a passive listener to answer what people ask . however, this passive and relatively simple response mechanism consumes the interest of human beings rapidly. |
| Approach: | They propose a model to transform a passive dialogue agent into a speaker by using external knowledge to express proactivity. |
| Outcome: | The proposed model obtains competitive results on the automatic and manual metrics and improves fluency and engagement of the chatbot significantly. |
Hooks in the Headline: Learning to Generate Headlines with Controlled Styles (2020.acl-main)
Copied to clipboard
| Challenge: | Current summarization systems only produce plain, factual headlines, far from the practical needs for exposure and memorableness of the articles. |
| Approach: | They propose a task to generate relevant headlines with three style options . they propose combining summarization and reconstruction tasks into a multitasking framework . |
| Outcome: | The proposed method outperforms the state-of-the-art summarization model by 9.68% . it can generate relevant, fluent headlines with humor, romance and clickbait . |
Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports (2020.acl-main)
Copied to clipboard
| Challenge: | Existing abstractive summarization models do not guarantee factual correctness of summaries . |
| Approach: | They propose a framework where models evaluate factual correctness by fact-checking it against its reference using an information extraction module. |
| Outcome: | The proposed method significantly improves the factual correctness and overall quality of outputs over a competitive neural summarization system, producing radiology summaries that approach the quality of human-authored ones. |
ParaDetox: Detoxification with Parallel Data (2022.acl-long)
Copied to clipboard
Varvara Logacheva, Daryna Dementieva, Sergey Ustyantsev, Daniil Moskovskiy, David Dale, Irina Krotova, Nikita Semenov, Alexander Panchenko
| Challenge: | a pipeline for the collection of parallel data for the detoxification task is available. |
| Approach: | They propose a pipeline for the collection of parallel data for the detoxification task . they collect non-toxic paraphrases for over 10,000 English toxic sentences . |
| Outcome: | The proposed pipeline outperforms state-of-the-art models on both automatic and manual evaluations. |
Summarization of Opinionated Political Documents with Varied Perspectives (2025.coling-main)
Copied to clipboard
| Challenge: | Political ideologies can lead people to develop misperceptions of groups with opposing opinions, such as the 2024 US presidential election, French legislative election, or the Brexit referendum. |
| Approach: | They propose a dataset and task for independently summarizing political perspectives in a set of opinionated news articles. |
| Outcome: | The proposed dataset and task evaluates models of varying sizes and architectures on a set of opinionated news articles. |
Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Tasks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) evaluation is a patchy and inconsistent landscape . established automatic evaluation metrics are poor surrogates, correlating weakly with human judgement. |
| Approach: | They propose to use both automatic and human evaluation to evaluate generative LLMs on three NLP benchmarks: text summarisation, text simplification and grammatical error correction. |
| Outcome: | The proposed model outperforms many popular models according to human reviewers on the majority of metrics, while scoring much worse when using classic automatic evaluation metrics. |
Multi-Source Multi-Type Knowledge Exploration and Exploitation for Dialogue Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models focus on identifying specific types of dialogue knowledge and utilizing corresponding datasets for training, but lack generalization capabilities and computational resources. |
| Approach: | They propose a framework that explores multi-source multi-type knowledge from LLMs by leveraging diverse datasets and exploits it for response generation. |
| Outcome: | The proposed framework exploits multi-source multi-type knowledge from LLMs to generate coherent, informative, and fluent responses. |
KoCoSa: Korean Context-aware Sarcasm Detection Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Sarcasm is a form of verbal irony where someone says the opposite of what they mean . misunderstanding this sarcasm may lead to fatal errors in dialogue systems . |
| Approach: | They propose a dataset for the Korean dialogue sarcasm detection task that uses 12.8K daily Korean dialogues and the labels on the last response. |
| Outcome: | The proposed system outperforms strong baselines like large language models in the Korean sarcasm detection task. |
What to Read in a Contract? Party-Specific Summarization of Legal Obligations, Entitlements, and Prohibitions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems that generate section-wise summaries of contracts can be tedious due to length and complexity of legalese. |
| Approach: | They propose a task of party-specific extractive summarization for legal contracts . they train a pairwise importance ranker and propose incorporating domain-specific notions of importance . |
| Outcome: | The proposed system generates a party-specific contract summary using a dataset of lease agreements and lease agreements. |
Multi-Objective Forward Reasoning and Multi-Reward Backward Refinement for Product Review Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Product review summarization aims to generate a concise summary based on product reviews . factual accuracy, aspect comprehensiveness, and content relevance are challenges . |
| Approach: | They propose an FB-Thinker framework to improve product review summarization ability . they propose two Chinese product review summary datasets for instruction-tuning and evaluation . |
| Outcome: | The proposed framework improves product review summarization with forward reasoning and backward refinement. |
Error Analysis of Multilingual Language Models in Machine Translation: A Case Study of English-Amharic Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual large language models have significantly advanced machine translation, yet challenges remain for low-resource languages like Amharic. |
| Approach: | They evaluated the performance of NLLB-200 and M2M in English-Amharic bidirectional translation using the Lesan AI dataset. |
| Outcome: | The proposed models outperformed the existing models in English-Amharic bidirectional translation using the Lesan AI dataset. |
Translate Smart, not Hard: Cascaded Translation Systems with Quality-Aware Deferral (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing quality estimation metrics are used to design effective deferral rules for machine translation. |
| Approach: | They propose a simple yet effective approach for machine translation using existing quality estimation metrics as deferral rules. |
| Outcome: | The proposed approach outperforms existing models in large translation tasks while reducing computational costs. |